Skip to content

Give the Python runners one canonicaliser, and a float policy that is stated - #2

Merged
StepenkoAnatoli merged 1 commit into
mainfrom
fix/shared-python-conformance
Sep 20, 2026
Merged

StepenkoAnatoli merged 1 commit into
mainfrom
fix/shared-python-conformance

Conversation

@StepenkoAnatoli

Copy link
Copy Markdown
Owner

The defect

Three Python conformance runners each carried their own canonicaliser. Two were hand-written byte-for-byte alike; the third reached for json.dumps(sort_keys=True), which is a different algorithm. The divergence was real and provable:

{"a": 1.0}    Node -> {"a":1}     015abd7f...      fi -> {"a":1.0}    c29a44ab...
{"a": -0.0}   Node -> {"a":0}     45b619e9...      fi -> {"a":-0.0}   952b7dc4...

Cross-language agreement held only because no vector in any packet contained a float. That is the whole defect. A suite comparing 28 chosen inputs does not establish that two functions agree; it establishes that they agree on 28 inputs. The tests were green and the claim was false.

What changed

File
bin/conformance_common.py new — the single implementation: canonicalisation, ECMAScript number formatting, JSON string escaping, duplicate-key-safe parsing, shared packet preflight, report rendering. Standard library only.
bin/ledger_conformance.py −1 copy; binds float_policy="reject"
bin/property_vector_conformance.py −1 copy; binds float_policy="reject"
bin/fi_sidecar_conformance.py −1 copy; binds float_policy="normalize"
lib/ledger-conformance.mjs new vector kind canonical-float
conformance/qualification-ledger-vectors.json +13 vectors (QL-F-01…QL-F-13)
test/canonical-float-policy.test.mjs new, 6 tests
.gitignore __pycache__/, *.pyc
docs/ARCHITECTURE.md the new boundary; inventory recounted

The float policy is now an argument, not an accident

canonical_json(value, float_policy=...) is required, and an unrecognised value raises. The two policies were both deliberate and previously indistinguishable from a bug:

  • reject — ledger and property vectors carry hashes and chain positions. A float there means the packet is wrong, and refusing says more than producing a plausible digest for a nonsensical record.
  • normalize — FI vectors describe documents that may legitimately carry one.

js_number implements ECMAScript Number::toString directly rather than post-processing repr. Both languages produce shortest-round-trip digits, so the digits were never the problem — everything around them was, and each difference is a different SHA-256:

1.0 / 1      -0.0 / 0      1e-07 / 1e-7      1e+17 / 100000000000000000      1e-05 / 0.00001

Post-processing repr catches the first four and misses the fifth, because the two languages switch to exponential at different thresholds, not at different formats.

What the vectors can and cannot pin

The 13 new vectors declare floatPolicy explicitly, and only normalize is expressible. That limit is worth recording rather than working around: in JavaScript 1.0 is 1, so a Node runner parsing the packet has no float left to reject by the time it sees the value. A reject vector would pass on the Node side for a reason unrelated to the policy it claims to test.

So the reject policy is a Python-side property, tested directly in test/canonical-float-policy.test.mjs, which hands real Python floats to the module instead of routing them through JSON. That file also compares the two canonicalisers as functions over inputs nobody put in a packet, and asserts structurally that no runner has grown a fourth private copy.

Verification

node research-kit/bin/selftest.mjs    -> 565 passed, 0 failed
node research-kit/bin/preflight.mjs   -> PASS, 0 blocking, 0 warnings
ledger    node 23 / python 23   agree per vector, all PASS
property  node  5 / python  5   agree per vector, all PASS
fi        node 13 / python 13   agree per vector, all PASS

The guards were tested by reproducing the drift, twice, because the first reproduction was incomplete:

injected drift caught by
repr() instead of Number::toString QL-F-01,06,07,08,09,12 red; 2 of 6 unit tests red
repr() for negative zero QL-F-02,12 red

The second run exists because the first injected the drift after the if value == 0 short-circuit, so QL-F-02 stayed green and the negative-zero path was still unproven. A regression test that has never seen the regression is a guess.

Things I got wrong along the way

  • ConformanceError first subclassed Exception, which broke the FI runner — its domain logic catches ValueError around a single document because a malformed one is a vector result. It subclasses ValueError now, and the docstring says why.
  • The first float test sent cases through JSON.stringify, so {a: 1.0} arrived in Python as the integer 1 — the test for the float defect was not exercising a float. Cases now travel as tagged markers.
  • Its guard was then wrong in the opposite direction, asserting the decoded value was a float in JavaScript, which is not a question JavaScript can answer. The guard runs in Python now.
  • Sharing the module introduced a byproduct that did not exist before: importing a sibling makes CPython write bin/__pycache__/. Unignored, every conformance run would dirty the working tree — the defect f9f78f1 just removed. Ignored in the same commit that causes it.

Scope

Deliberately excluded, per the request: no release-validator refactor, no unrelated-surface removal.

🤖 Generated with Claude Code

… stated

WHAT CHANGED
  bin/conformance_common.py              new, shared: canonicalisation, ECMAScript
                                         number formatting, JSON string escaping,
                                         duplicate-key-safe parsing, packet preflight
  bin/ledger_conformance.py              -1 copy; binds float_policy="reject"
  bin/property_vector_conformance.py     -1 copy; binds float_policy="reject"
  bin/fi_sidecar_conformance.py          -1 copy; binds float_policy="normalize"
  lib/ledger-conformance.mjs             new vector kind: canonical-float
  conformance/qualification-ledger-vectors.json   +13 vectors (QL-F-01..13)
  test/canonical-float-policy.test.mjs   new, 6 tests
  .gitignore                             __pycache__/, *.pyc
  docs/ARCHITECTURE.md                   the new boundary; inventory recounted

WHY
  Three runners each carried their own canonicaliser. Two were hand-written byte
  for byte alike; the third reached for json.dumps(sort_keys=True), which is a
  different algorithm. The divergence was real and provable:

    {"a": 1.0}   Node {"a":1}   015abd7f...    fi {"a":1.0}   c29a44ab...
    {"a": -0.0}  Node {"a":0}   45b619e9...    fi {"a":-0.0}  952b7dc4...

  Cross-language agreement held only because no vector in any packet contained a
  float. That is the whole defect. A suite comparing 28 chosen inputs does not
  establish that two functions agree; it establishes that they agree on 28
  inputs. The tests were green and the claim was false.

THE FLOAT POLICY IS NOW AN ARGUMENT, NOT AN ACCIDENT
  canonical_json(value, float_policy=...) is required and unrecognised values
  raise, because the two policies here are both deliberate and were previously
  indistinguishable from a bug:

    reject      ledger and property vectors carry hashes and chain positions.
                A float there means the packet is wrong, and refusing says more
                than producing a plausible digest for a nonsensical record.
    normalize   FI vectors describe documents that may legitimately carry one.

  js_number implements ECMAScript Number::toString directly rather than
  post-processing repr. Both languages produce shortest-round-trip digits, so
  the digits were never the problem - everything around them was, and each
  difference is a different SHA-256:

    1.0 / 1      -0.0 / 0      1e-07 / 1e-7      1e+17 / 100000000000000000
    1e-05 / 0.00001

  Post-processing repr catches the first four and misses the fifth, because the
  two languages switch to exponential at different THRESHOLDS, not at different
  formats. All 24 float cases now match Node byte for byte.

WHAT THE VECTORS CAN AND CANNOT PIN
  The 13 new vectors declare floatPolicy explicitly, and only "normalize" is
  expressible. That limit is worth recording rather than working around: in
  JavaScript 1.0 IS 1, so a Node runner parsing the packet has no float left to
  reject by the time it sees the value. A "reject" vector would pass on the Node
  side for a reason unrelated to the policy it claims to test.

  So the reject policy is a Python-side property, tested directly in
  test/canonical-float-policy.test.mjs, which hands real Python floats to the
  module instead of routing them through JSON. That file also compares the two
  canonicalisers as FUNCTIONS over inputs nobody put in a packet, and asserts
  structurally that no runner has grown a fourth private copy.

WHAT I VERIFIED
  node research-kit/bin/selftest.mjs    -> 565 passed, 0 failed
  node research-kit/bin/preflight.mjs   -> PASS, 0 blocking, 0 warnings
  both ledger runners, per vector       -> 23/23 agree, all PASS
  ledger 10 / fi 13 / property 5        -> Node and Python agree per vector

  The guards were tested by REPRODUCING the drift, twice, because the first
  reproduction was incomplete:

    repr() instead of Number::toString  -> QL-F-01,06,07,08,09,12 red, 2 of 6
                                           unit tests red
    repr() for negative zero            -> QL-F-02,12 red

  The second run exists because the first injected the drift AFTER the
  `if value == 0` short-circuit, so QL-F-02 stayed green and the negative-zero
  path was still unproven. A regression test that has never seen the regression
  is a guess.

WHAT I GOT WRONG AND FIXED
  ConformanceError first subclassed Exception. That broke the FI runner, whose
  domain logic catches ValueError around a single document because a malformed
  one is a VECTOR RESULT - the packet carries a document with a duplicate fiId
  whose whole purpose is to be rejected and recorded as FAIL. A bare Exception
  escaped those handlers and turned an expected failure into a crash. It
  subclasses ValueError now, and the docstring says why.

  The first version of the float test sent cases through JSON.stringify, so
  {a: 1.0} arrived in Python as the integer 1. The test for the float defect was
  not exercising a float and passed for that reason. Cases now travel as tagged
  markers rebuilt on both sides from the same spec.

  Its guard was then wrong in the opposite direction: it asserted the decoded
  value was a float in JAVASCRIPT, which is not a question JavaScript can answer
  - Number('1.0') === 1. The guard runs in Python now, which is the side that
  actually distinguishes them.

  Sharing the module introduced a byproduct that did not exist before: importing
  a sibling makes CPython write bin/__pycache__/. Unignored, every conformance
  run would dirty the working tree - the defect f9f78f1 just removed. Ignored in
  the same commit that causes it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@StepenkoAnatoli
StepenkoAnatoli merged commit 5150c23 into main Sep 20, 2026
1 check passed
@StepenkoAnatoli
StepenkoAnatoli deleted the fix/shared-python-conformance branch September 20, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant